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DETAILED ACTION 

1 . The application filed April 23, 2004 is presented for examination and claims 1-27 
are pending. 

Specification 

2. The abstract of the disclosure is objected to because it does not have the proper 
content of the disclosure. Correction is required. See MPEP § 608.01(b). 

Claim Rejections - 35 USC §112 

3. The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

Claims 14 and 17 are rejected under 35 U.S.C. 1 12, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. 

Claim 14 recites the limitation "the selection" in line 3. There is insufficient 
antecedent basis for this limitation in the claim. Correction is required. 

Claim 17 recites the limitation "the predefined button" in line 4. There is 
insufficient antecedent basis for this limitation in the claim. Correction Is required. 

Ciaim Rejections - 35 USC § 102 

4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 
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A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

Claims 1-6, 8 and 14-18, 20, 26, and 27 are rejected under 35 U.S.C. 102(b) as 
being unpatentable over US Patent No. 6,512,838 by Rafii et al. 

Regarding claim 1 , Rafii et al. teach a 3D-input device for inputting information 
using a virtual keyboard (a three-dimensional sensor system 10 comprising a three- 
dimensional sensor 20 focused essentially edge-on towards the fingers 30 of a user's 
hands 40, as the fingers "type" on a substrate 50, shown here atop a desk or other work 
surface 60 (column 7, lines 62-66) wherein substrate 50 and/or template 70 may be 
referred to herein as a virtual keyboard or virtual device for inputting digital data and/or 
commands (column 8, lines 18-20)), comprising: 

a hand position and finger order detennination unit that determines (routine 285 
uses Z-axis distance measurements to determine position of the fingers with respect to 
the rows of the virtual kevboard. e.g., distance Z1 or Z2 in FIG. 1 A (column 18, lines 28- 
31 )): a selected button, of a plurality of buttons of the virtual keyboard, that is selected 
by a user (a user's finger having the smallest Y-axis position or the greatest downward 
velocity is selected as the key entry finger, e.g., the finger that will strike one of the 
virtual keys on the virtual data input device (column 19, lines 10-13)); and an order of 
the user's fingers used to select the selected button (the user can command the 
companion device to position the virtual keyboard or other input device in the proper 
starting position. For instance, if the user typically begins to key by placing the right 
hand fingers on home row J, K, L, and ":" keys, and the left fingers on F, D, S and A 
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keys, the software will move the keys of the virtual keyboard to such a position (column 
18, lines 51-56)): 

a key information storage unit (memory and software Fig.3 (280 and 285)) that 
stores key values respectively mapped to both a predefined button of the plurality of 
buttons of the virtual keyboard and a predefined order of the user's fingers used to 
select the predefined button (routine 285 in essence moves or relocates the virtual 
keyboard to under the user's fingers. Such procedure may be carried out by mapping 
the image obtained from sensor 20 to the fingers of the template, and then mapping the 
touched keys to the natural position for the user, which natural position was determined 
during the template construction phase (column 21 , lines 35-41)); and 

a key determination unit (software routine Fig.3 (285)) that finds a selected key 
value by matching the selected button and order of the user's fingers with the 
predefined button and predefined order of the user's fingers mapped in the key 
information storage unit (routine 285 in essence moves or relocates the virtual keyboard 
to under the user's fingers. Such procedure may be carried out by mapping the image 
obtained from sensor 20 to the fingers of the template, and then mapping the touched 
keys to the natural position for the user, which natural position was detemiined during 
the template construction phase (column 21 . lines 35-41 )) wherein the software 285 is a 
code or routine which works with the memory 280 to store different finger positions of 
the user's hand. 

Regarding claim 2, Rafii et al. teach the key determination unit (software routine 
Fig.3 (285)) outputs the selected key value (routine 285 in essence moves or relocates 
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the virtual l<eyboard to under tfie user's fingers. Such procedure may be carried out by 
mapping the image obtained from sensor 20 to the fingers of the template, and then 
mapping the touched keys to the natural position for the user, which natural position 
was determined during the template construction phase (column 21 , lines 35-41)) where 
the software or routine outputs the selected key value by mapping the touched keys to 
natural position for the user. 

Regarding claim 3, Rafii et al. teach a sensing device that senses a user's finger 
movements (a keystroke can be sensed as commencing with a detected finger up 
movement followed by a quick finger down motion (column 19, lines 8-10)); and 

a signal processing unit (CPU or controller Fig.3 (260)) that processes a signal 
output from the sensing device to detect the movement of the user's fingers(a keystroke 
can be sensed as commencing with a detected finger up movement followed by a quick 
finger down motion (column 19, lines 8-10)), wherein the hand position and finger order 
determination unit utilizes information output by the signal processing unit to determine 
the selected button and the order of the user's fingers (routine 285 uses Z-axis distance 
measurements to determine position of the fingers with respect to the rows of the virtual 
keyboard, e.g., distance Z1 or Z2 in FIG. 1A (column 18, lines 28-31)). 

Regarding claim 4, Rafii et al. teach the sensing device (three-dimensional sensor 
Fig.1 A (20) comprises a plurality of sensors (rays Fig.1 A (140) arranged individually on 
a user's fingers Fig.1 A (30) wherein the sensor (20) comprises a plurality of rays (140) 
which represents a plurality of sensors. 

Regarding claim 5, Rafii et al. teach in the key information storage unit (memory 
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and software Fig. 3 (280 and 285)), key values are allocated to each of the plurality of 
buttons of the virtual keyboard based upon the number of sensors ( Sensor 20 is aimed 
along the Z-axis to determine which of the user's finger tips 30 touch what portions of 
template 70, e.g., touch which virtual keys, in what time order (column 11, lines 3-6) 
wherein the number of portions touched of template 70 represents the number of 
sensors. 

Regarding claim 6, Rafii et al. teach the virtual buttons are arranged so that the 
key values are ordered by frequency of use (Turning now to operation of three- 
dimensional sensor 20. the sensor emits radiation of a known freauencv and detects 
energy returned by surfaces of objects within the optical field of view (column 10, line 66 
to column 1 1 , line 2) wherein the sensor 20 emits radiation only when there is a number 
of keys being striped and depending on how many times a key or button is being 
pressed is the frequency of use. 

Regarding claim 8, Rafii et al. teach the virtual buttons include key values that 
are defined by the user (When a key is actually struck (as perceived by the user's finger 
movement), the struck key may be highlighted using a different color or contrast. If the 
virtual keys are not in a correct rest position, the user can command the companion 
device to position the virtual keyboard or other input device in the proper starting 
position. For instance, if the user typically begins to key by placing the right hand fingers 
on home row J. K, L, and ":" keys, and the left fingers on F, D, S and A keys, the 
software will move the keys of the virtual keyboard to such a position (column 18, lines 
47-56) wherein depending on the user's finger position, the software (285) maps the key 
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value position to the virtual buttons. 

Regarding claim 14, Rafii et al. teach a 3D-lnput method for inputting infomiation 
using a virtual keyboard (a three-dimensional sensor system 10 comprising a three- 
dimensional sensor 20 focused essentially edge-on towards the fingers 30 of a user's 
hands 40, as the fingers "type" on a substrate 50, shown here atop a desk or other work 
surface 60 (column 7, lines 62-66) wherein substrate 50 and/or template 70 may be 
referred to herein as a virtual kevboard or virtual device for inputting digital data and/or 
commands (column 8, lines 18-20)) comprising: 

sensing the selection of a virtual button of the virtual keyboard by a user (the 
sensor Fig. la (20) detects the user's finger that touched the template (70) (virtual 
keyboard) (column 21, lines 37-41)); 

sensing positions of the user's fingers relative to the virtual button, and the order 
of the user's fingers that are used to select the virtual button (When a key is actually 
struck (as perceived by the user's finger movement), the struck key may be highlighted 
using a different color or contrast. If the virtual keys are not in a conrect rest position, the 
user can command the companion device to position the virtual keyboard or other input 
device in the proper starting position . For instance, if the user typically begins to key by 
placing the right hand fingers on home row J, K, L, and ":" keys, and the left fingers on 
F, D, S and A keys, the software will move the keys of the virtual keyboard to such a 
position (column 18, lines 47-56); and 

identifying a selected key value corresponding to the sensed positions of 
the fingers and the order of the user's fingers that are used to select the virtual button. 
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amongst a plurality of stored key values (templates preferably are used in the present 
invention to help identifv user finger positions from data obtained from sensor 20. 
Templates can assist classification algorithm (or classifier) 285 in distinguishing 
boundaries between fingers when discontinuities are not necessarily apparent. For 
example, in FIG. 7A, the third and fourth user's fingers (fingers 3 and 4) are relatively 
close together (column 21, lines 15-22). 

Regarding claim 15, Rafii et al. teach outputting the selected key value (routine 
285 in essence moves or relocates the virtual keyboard to under the user's fingers. 
Such procedure may be carried out by mappino the image obtained from sensor 20 to 
the fingers of the template, and then mapping the touched keys to the natural position 
for the user, which natural position was determined during the template construction 
phase (column 21, lines 35-41)) where the software or routine outputs the selected key 
value by mapping the touched keys to natural position for the user. 

Regarding claim 16, Rafii et al. teach sensing the selection of a virtual button 
comprises arranging a plurality of sensors individually on the user's fingers and 
detemrjining the position of those sensors relative to the virtual button (routine 285 in 
essence moves or relocates the virtual keyboard to under the user's fingers. Such 
procedure may be carried out by mapping the image obtained from sensor 20 to the 
fingers of the template, and then mapping the touched keys to the natural position for 
the user, which natural position was determined during the template construction phase 
(column 21 , lines 35-41 )) wherein the sensor 20 detects the position of the user's finger 
when touched and the routine 285 maps the image obtained from the rays of the sensor 
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to the fingers of the template. 

Regarding claim 17, Rafii et al. teach the plurality of stored key values are 
stored by mapping key values to respective predefined virtual buttons and a predefined 
order of the user's fingers used to select the predefined button (the user's fingers are 
placed in a typing position in the work surface in front of three-dimensional sensor 20, 
either on a virtual keyboard or simply on the work surface. This step is used to map the 
user fingers to the elements of the template and to calibrate the user's fingers to the 
keys of the virtual keyboard (or work surface) before a typing session starts (column 20, 
lines 27-33)) wherein the software or routine maps the predefined positions to the virtual 
keyboard. 

Regarding claim 18, Rafii et al. teach the virtual buttons are arranged so that the 
key values are ordered by frequency of use (Turning now to operation of three- 
dimensional sensor 20, the sensor emits radiation of a known frequency and detects 
energy returned by surfaces of objects within the optical field of view (column 10, line 66 
to column 1 1 , line 2) wherein the sensor 20 emits radiation only when there is a number 
of keys being striped and depending on how many times a key or button is being 
pressed is the frequency of use. 

Regarding claim 20, Rafii et al. teach the virtual buttons include key values that 
are defined by the user (When a key is actually struck (as perceived by the user's finger 
movement), the struck key may be highlighted using a different color or contrast. If the 
virtual keys are not in a correct rest position, the user can command the companion 
device to position the virtual keyboard or other Input device in the proper starting 
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position. For instance, if the user typically begins to key by placing the right hand fingers 
on home row J, K, L, and ":" keys, and the left fingers on F, D, S and A keys, the 
software will move the keys of the virtual keyboard to such a position (column 18, lines 
47-56) wherein depending on the user's finger position, the software (285) maps the key 
value position to the virtual buttons. 

Regarding claim 26, Rafii et al. teach a soft key (soft keyboard Fig.lB (105)) 
mapping method for mapping keys onto virtual buttons of a virtual keyboard (software or 
routine Fig.3 (285)) that are selected by a user's fingers upon which are individually 
mounted a plurality of sensors (the sensor (20) comprises a plurality of rays (150) 
projected on each finger represent a plurality of sensors), the method comprising: 

determining the number of sensors (the sensor Fig.lB (20) comprises a plurality 
of rays (1 50) projected on each finger represent the number of sensors)); 

allocating key values according to the number of sensors and mapping the 
allocated key values onto a first virtual button (the classifier uses this template to quickly 
map image in acquired frames to each user's fingers. As part of the template 
construction, preferably a mapping of the positions of the user's fingers to specific 
keyboard keys at a rest position is defined. For instance, routine 285 and CPU 270 can 
instruct the companion device 80 that, at rest, the user's left hand fingers touch the :"A", 
"S", "D" and "F" keys, and the user's right hand fingers touch the "J", "K", "L", and ":" 
keys (column 20, lines 4-13)); and 

repeating the determining, allocating and mapping for the remaining virtual 
buttons (three-dimensional sensor 20 will be repeatediv capturing the contour map of 
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the user's fingers. The data thus captured will be placed, e.g., by software 285 in a table 
or matrix such as shown In FIGS. 7A-70 (column 20, lines 34-37)). 

Regarding claim 27, Rafii et al. teach a virtual keyboard comprising a plurality 
of virtual buttons selectable by a user's fingers upon which are mounted a plurality of 
sensors, the virtual keyboard constructed by mapping key values onto each of the 
virtual buttons and arranging the virtual buttons according to a predetermined condition 
using a method (routine 285 in essence moves or relocates the virtual keyboard to 
under the user's fingers. Such procedure may be carried out by mapping the image 
obtained from sensor 20 to the fingers of the template, and then mapping the touched 
keys to the natural position for the user, which natural position was determined during 
the template construction phase (column 21 , lines 35-41)) wherein the software 285 is a 
code or routine which works with the memory 280 to store different finger positions of 
the user's hand) comprising: 

determining the number of sensors (the sensor Fig. IB (20) comprises a plurality 
of rays (150) projected on each finger represent the number of sensors)); 

allocating key values according to the number of sensors and mapping the 
allocated key values onto a first virtual button (the classifier uses this template to quickly 
map image in acquired frames to each user's fingers. As part of the template 
construction, preferably a mapping of the positions of the user's fingers to specific 
keyboard keys at a rest position is defined. For instance, routine 285 and CPU 270 can 
instruct the companion device 80 that, at rest, the user's left hand fingers touch the :"A", 
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"S", "D" and "F" keys, and the user's right hand fingers touch the "J". "K". "L", and ":" 
keys (column 20, lines 4-13)); and 

repeating the determining, allocating and mapping for the remaining virtual 
buttons (three-dimensional sensor 20 will be repeatedly capturing the contour map of 
the user's fingers. The data thus captured will be placed, e.g., by software 285 in a table 
or matrix such as shown in FIGS. 7A-70 (column 20, lines 34-37)). 

Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 7. 9-11, 19. 21-23 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over US Patent No. 6,512,838 ("Rafii et al.") in view of US Patent 
Application No. 20050104869 ("Chung"). 

Regarding claims 7 and 19, Rafii et al. teach all the claimed limitations with the 
exception of providing the virtual buttons are arranged so that the key values are in 
alphabetical order. 

However, Chung teaches as shown in FIG. lb, "2" key has letters "ABC" 
imprinted thereon. Each number key has letters imprinted thereon, respectively. The 
alphabet is grouped into sets of 3 letters according to the alphabetical order. 
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Thus, it would have been obvious to a person of ordinary skill in the art at the 
time the invention was made to utilize the key "2" that has the letters "ABC" as taught by 
Chung in the three-dimensional virtual keyboard disclosed by Rafii et al. because this 
would provide a method for character entry that the desired character can be entered 
separately on each of a real kevpad entry mode and a virtual keypad entry mode 
according to the prescribed way although the identical key is pressed in each of both 
modes, by establishing separately the entry mode of a virtual kevoad that maintains the 
state of an independent entry mode regardless of the entry mode of a real kevpad and 
the key presses on a real keypad (paragraph 0017). 

Regarding claims 9 and 21, Chung teaches each virtual button comprises two 
key values ("Q" and "Z" is allocated to "1" jsey due to the low frequency of use (Fig.1 b) 
wherein the key "1" has two key values. Same motivation as claim 7. 

Regarding claims 10 and 22, Chung teaches each virtual button comprises three 
key values (as shown in FIG. lb, "2" key has letters "ABC" imprinted thereon. Each 
number key has letters imprinted thereon, respectively. The alphabet is grouped into 
sets of 3 letters according to the alphabetical order)wherein the Key "2" has three letter 
values. Same motivation as claim 7. 

Regarding claims 1 1 and 23, Chung teaches each virtual button comprises four 
key values (in Fig.1 c of the prior art, key "9" has a four key values (WXYZ). Same 
motivation as claim 7. 
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6. Claims 12, 13, 24, and 25 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over US Patent No. 6,512,838 ("Rafil et al.") in view of US Patent 
Application No. 20030193478 ("Ng et al."). 

Regarding claims 12, 13, 24, and 25, Rafii et al. teach all the claimed limitations 
with the exception of providing each virtual button comprises five key values and six key 
values. 

However, Ng et al. teach a visual mapping reduced keyboard Fig.1 (10) wherein 
keyl has 6 characters mapped to it (paragraph 0064) and key4 has 5 characters 
mapped to it (paragraph 0067). 

Thus, it would have been obvious to a person of ordinary skill in the art at the 
time the invention was made to utilize the five and six key values as taught by Ng et al. 
into the three-dimensional virtual keyboard disclosed by Rafii et al. because this would 
provide each key having at least one feature wherein the feature is a data value 
(paragraph 0010). 

Conclusion 

7. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jean Lesperance whose telephone number is (571) 
272-7692. The examiner can normally be reached on from Monday to Friday between 
10:OOAM and 6:30PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Richard Hjerpe, can be reached on (571) 272-7691. 
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Any response to this action should be mailed to: 

Commissioner of Patents and Trademarks 
Washington, D.C. 20231 
or faxed to: 

(571 ) 273-8300 (for Technology Center 2600 only) 
Hand-delivered responses should be brought to Crystal Park II, 2121 Crystal 

drive, Arlington, VA, Sixth Floor (Receptionist). 

Any inquiry of a general nature or relating to the status of this application or 

proceeding should be directed to the technology Center 2600 Customer Service Office 

whose telephone number is (703) 306-0377. 

Jean Lesperance 

Art Unit 2629 

Date 3/1/2007 




